Skip to content

[SQSERVICES-1825] OAuth refresh token generation (6/n)#2985

Closed
battermann wants to merge 37 commits intoSQSERVICES-1780-be-oauth-o-auth-token-handling-and-validation-2from
SQSERVICES-1825-be-oauth-refresh-token-generation
Closed

[SQSERVICES-1825] OAuth refresh token generation (6/n)#2985
battermann wants to merge 37 commits intoSQSERVICES-1780-be-oauth-o-auth-token-handling-and-validation-2from
SQSERVICES-1825-be-oauth-refresh-token-generation

Conversation

@battermann
Copy link
Contributor

@battermann battermann commented Jan 13, 2023

https://wearezeta.atlassian.net/browse/SQSERVICES-1825

Checklist

  • Add a new entry in an appropriate subdirectory of changelog.d
  • Read and follow the PR guidelines

isovector and others added 30 commits December 27, 2022 10:48
* chore: update hackage pins to use new polysemy
* chore: also pin kind-generics
* chore: changelog
* feat: track federation api calls
* chore: make format
* fix: give a default instance for other packages
* feat: galley callsfed tracking
* chore: make format
* fix: cargohold
* chore: make format
* doc: changelog.d
* chore: remove spurious HasCallStack
* doc: changelog.d
Executing `"$@"` within first argument of `if` prevents `set -e` from
immediately failing the whole script.

It could also be written as `while ! "$@"; do ...`, but then getting status of
`"$@"` is more complicated as `! "$@"` has status=0 and overwrites the value of
`$?`.
Path style is not supported for newer buckets, more info:
https://aws.amazon.com/blogs/aws/amazon-s3-path-deprecation-plan-the-rest-of-the-story/

All object storage providers (like MinIO, ScalityRing, etc) might not work with
vhost style addressing, so this change introduces a new configuration option in
cargohold as aws.s3AddressingStyle to choose the addressing style.

Other changes:

* Move wire-server from using forked version of amazonka to upstream HEAD.

The option to choose S3 Addressing Style has been implemented in
brendanhay/amazonka#832

* Makefile: Skip schema migrations for packages without DB

This allows running something like `make ci package=cargohold` even if cargohold
doesn't produce a cargohold-schema executable.
* feat: track federation api calls
* chore: make format
* fix: give a default instance for other packages
* feat: galley callsfed tracking
* chore: make format
* fix: cargohold
* chore: make format
* doc: changelog.d
* feat: MakesFederatedCall servant combinator
* chore: make format
* doc: haddock
* fix: add RoutesToPaths instance
* feat: use updated extension point for MakesFederatedCall
* chore: make format
* chore: remove spurious HasCallStack
* feat: add some federated calls to brig
* feat: federated calls in brig/client API
* feat: more api calls
* fix: add callsFed
* feat: finish adding MakesFederatedCall documentation
* chore: make format
* feat: cargohold api
* Add changelogs
* Fix compilation of integration tests in Brig
* Revert "Fix compilation of integration tests in Brig"
This reverts commit 2310a32.
* fix: clean up brig integration test callsites
* feat: SolveCallsFed for variadic numbers of callsFed
* feat: track federation api calls
* chore: make format
* fix: give a default instance for other packages
* feat: galley callsfed tracking
* chore: make format
* fix: cargohold
* chore: make format
* doc: changelog.d
* feat: MakesFederatedCall servant combinator
* chore: make format
* doc: haddock
* fix: add RoutesToPaths instance
* feat: use updated extension point for MakesFederatedCall
* chore: make format
* chore: remove spurious HasCallStack
* feat: add some federated calls to brig
* feat: federated calls in brig/client API
* feat: more api calls
* fix: add callsFed
* feat: finish adding MakesFederatedCall documentation
* chore: make format
* feat: cargohold api
* Add changelogs
* Fix compilation of integration tests in Brig
* Revert "Fix compilation of integration tests in Brig"
This reverts commit 2310a32.
* fix: clean up brig integration test callsites
* feat: patch internal API
* feat: conversation API
* feat: many more galley apis
* feat: finish API porting
* fix: integration tests
* doc: changelog
* feat: SolveCallsFed for variadic numbers of callsFed
* feat: remove extaneous calls to callsFed
* chore: separate out ApplyMods
* doc: remove invalid scheme from example socks5 host
* doc: make example YAML config for deeplink configs a valid YAML file
Co-authored-by: Stefan Matting <stefan@wire.com>
Co-authored-by: Marko Dimjašević <marko.dimjasevic@wire.com>
* charts/coturn: add TLS cipher configuration, comply with BSI TR-02102-2 by default.

* changelog: update.
Co-authored-by: Leif Battermann <leif.battermann@wire.com>
* galley: Avoid duplicate work for conversation-created notificiations

For every member in the conversation, the view of the conversation differs as
they see their own member metadata in detail which other memebrs' metadata is
not so detailed. When a conversation is created, each member of the conversation
gets their own view of the convesation in a notification. The
`convRemoteMembers` and `convLocalMembers` fields are mostly repeated across all
of these views. This commit avoids computing these lists from scratch n times,
where n is the number of members in the conversation.

* Galley.Intra.Push.Internal: Remove StrictData langauge pragma

The pragma causes everything inside `PushTo` to be strict. This causes larger
pushes (e.g. a new Conversation notification to n conversation members) to
allocate a lot of thunks internally for the `pushJson` field way before they are
actually needed causing spike in memory usage.

* Galley.Intra.Push.Internal: Refactor logic to chunk pushes

Using `foldr` to create chunks caused each list in `[[Gundeck.Push]]` to get
allocated and kept alive until it was completed consumed, causing memory spikes.
This combined with `mapConcurrently` would cause all of the `Gundeck.Push`
objects to get allocated almost at once before they were all consumed.

The refactored logic chunks `PushTo a` without using `foldr` and instead
implements `chunk` so that it creates the chunks lazily while ensuring that
elements of each chunk are also computed lazily.

* Galley.Intra.Push.Internal: Use chunked encoding to push notifications

This ensures that http-client doesn't try to compute `Content-Length` of a very
big JSON which forces the JSON bytestring to get computed and hence allocates a
lot of memory.
`-with-rtsopts=-N1` was set very long time ago when brig started depending on
http-client-openssl. It doesn't seem relevant anymore and using multiple cores
should improve performance.
* Force V2 when serialising conversations in events

* Update golden tests
* docs: add documentation of client id to zauth readme

* docs: make tag specification consistent
Release 2023-01-12 - (expected chart version 4.30.0)
Master->Develop after release
integrate post-install in the install guide
* fedcalls cli tool.

* Cleanup

* Output both dot and csv.

* Fixup

* Fixup

* Changelog.

* Fixup

* Fixup

* Update tools/fedcalls/README.md
@battermann battermann temporarily deployed to cachix January 13, 2023 14:01 — with GitHub Actions Inactive
@battermann battermann temporarily deployed to cachix January 13, 2023 14:01 — with GitHub Actions Inactive
@fisx fisx mentioned this pull request Jan 16, 2023
2 tasks
@battermann battermann temporarily deployed to cachix January 16, 2023 11:01 — with GitHub Actions Inactive
@battermann battermann temporarily deployed to cachix January 16, 2023 11:02 — with GitHub Actions Inactive
fisx added 2 commits January 16, 2023 12:03
…-oauth-refresh-token-generation

Conflicts:
charts/brig/templates/configmap.yaml
charts/brig/values.yaml
docs/src/developer/reference/config-options.md
docs/src/how-to/install/index.rst
libs/wire-api/src/Wire/API/Routes/Public/Brig.hs
nix/haskell-pins.nix
services/brig/src/Brig/API/Internal.hs
services/brig/src/Brig/API/Public.hs
services/brig/src/Brig/Options.hs
…oauth-refresh-token-generation' into SQSERVICES-1825-be-oauth-refresh-token-generation
@fisx fisx temporarily deployed to cachix January 16, 2023 11:05 — with GitHub Actions Inactive
@fisx fisx temporarily deployed to cachix January 16, 2023 11:05 — with GitHub Actions Inactive
@fisx fisx temporarily deployed to cachix January 16, 2023 11:05 — with GitHub Actions Inactive
@fisx fisx temporarily deployed to cachix January 16, 2023 11:05 — with GitHub Actions Inactive
@battermann
Copy link
Contributor Author

obsolete, replaced by #2989

@battermann battermann closed this Jan 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants